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FOREWORD 


One  of  the  research  goals  of  the  Decision  Sciences  Laboratory  is  the 
development  of  design  principles  for  automated  training  subsystems  which 
could  be  built -into  future  Information  Systems.  Such  subsystems  would 
provide  Information  Systems  with  the  capability  of  training  automatically 
their  own  operators.  The  need  for  such  on-the-job  training  has  already 
become  apparent.  To  be  able  to  design  such  a  capability  requires  first 
the  solution  of  many  conceptual  and  experimental  problems. 

Task  768204,  Automated  Training  for  Information  Systems,  under 
Project  7682,  Man-Computer  Information  Processing,  was  established 
to  formulate  and  answer  some  of  these  problems. 

This  report  is  one  in  a  planned  series  supporting  Task  768204.  It 
was  prepared  as  a  joint  effort  of  the  staff  of  Bio-Dynamics,  Inc.  ,  work¬ 
ing  under  terms  of  Contract  AF  1 9( 628) -455 ,  and  Decision  Sciences  Lab¬ 
oratory,  Electronic  Systems  Division.  The  authors  are  Drs.  Thomas  B. 
Sheridan  and  Benjamin  C.  Duggar  of  Bio-Dynamics,  Inc.  ,  and  Dr.  Sylvia  R. 
Mayer,  Contract  Monitor. 

Ms.  S.  Molner,  R.  Rosenberg,  J.  Mickunas,  and  Dr.  A.  Johnson  of 
Bio-Dynamics,  Inc.  are  contributing  directly  to  the  research  activities 
which  provide  a  background  for  development  of  the  general  principles 


described  in  this  report. 


Design  and  Use  of  Information  Systems 


for  Automated  on-the-Job  Training 
II.  Design  of  Self -Instructional  Features 


Abstract 


This  report  is  concerned  with  human  engineering  factors  in  the  design 
of  Information  Systems.  In  particular  it  is  addressed  to  the  design  of 
self -instructional  features  for  these  systems.  It  describes  theories, 
methodology,  and  design  principles  for  implementation  of  self- instructional 
features.  The  design  principles  were  induced  from  the  exploratory  research 
on  laboratory  models  of  Information  Systems  which  is  reported  in  Volume  I 
of  this  series,  from  studies  on  current  Information  Systems,  and  from  a 
literature  review. 

The  operational  concepts  underlying  this  study  are  stated,  and  an 
equipment  design  philosophy  is  proposed  to  complement  this  operational 
concept. 
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I.  INTRODUCTION 


This  report  outlines  some  general  design  principles  relating  to  the 
implementation  of  self-instructional  features  in  future  Information  Sys¬ 
tems.  These  principles  were  derived  from  study  of  existing  Information 
Systems,  a  literature  review,  and  from  a  series  of  exploratory  studies  on 
Information  System  models  which  were  described  in  a  previous  report  (1). 

A.  The  Air  Force  Requirement 

The  proposed  design  philosophy  of  this  study  is  dictated  by  the 
planned  operational  concept  for  advanced  Information  Systems.  These 
future  computerized  systems  will  serve  as  real-time,  problem-solving 
and  decision  aids  for  staff  and  command-level  users.  As  such  they  must 
provide  maximum  flexibility  in  operation  to  the  extent  that  the  problem 
solver  can  instantly  re-program  them. 

This  operational  concept  implies  that  the  user  must  have  extensive 
information  on  the  workings  and  capabilities  of  the  hardware  and  software 
aspects  of  the  equipment.  In  turn,  this  fact  poses  a  requirement  for 
training  aids  and  performance  supports  continuously  available  at  the  man- 
machine  interface.  It  also  demands  the  use  of  the  operational  equipment 
for  on-the-job  and  proficiency  training  in  the  operation  of  the  equipment. 

The  future  Information  System  user  will  not  be  the  traditional 
"equipment  operator"  who  solves  a  limited  number  of  well-defined  prob¬ 
lems  in  a  prescribed  manner.  He  will  be  a  specialist  in  some  Air  Force 
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mission  area.  He  will  use  the  Information  System  on  an  intermittent  basis 
to  solve  a  broad  range  of  dynamic  problems  which  demand  creative  methods 
of  attack  -  methods  which  cannot  be  pre-specified. 

This  operational  concept  and  the  new-style  user  it  implies  im¬ 
pose  the  requirement  for  a  new  approach  to  time-tested  military  solutions 
for  the  training  of  equipment  users. 

B.  Assumptions  Underlying  Traditional  Approach  to  Equipment  Design 

The  assumption  underlying  traditional  equipment  design  is  that  the 
equipment  will  be  used  only  by  the  completely  proficient.  Traditionally, 
the  bare  minimum  (or  less)  of  cues  for  operation  are  provided  at  the  man- 
machine  interface.  The  user  is  expected  to  have  all  required  skills  and 
information  for  operation  in  his  repertoire. 

The  activities  which  lead  up  to  the  ultimate  state  of  proficiency 
are  viewed  as  belonging  to  a  unique  type  of  behavior  -  "training"  -  which 
is  regarded  as  totally  different  from  "job  performance.  "  Furthermore, 
training  and  job  performance  are  assumed  each  to  require  different  types 
of  equipment  support. 

C.  Assumptions  Underlying  Proposed  Approach  to  Equipment  Design 

A  different  set  of  assumptions  underlies  the  design  philosophy 

proposed  here  for  advanced  Information  Systems.  It  is  proposed  that  equip¬ 
ment  be  designed  to  support  operation  by  a  broad  continuum  of  users  from 
the  newly-assigned  who  understands  little  about  the  equipment  (but  much 
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about  the  operational  problem)  to  the  fully  proficient.  Furthermore,  indi¬ 
vidual  users  can  be  expected  to  fluctuate  widely  over  time  in  their  profi¬ 
ciency.  Thus,  the  equipment  should  supply  maximum  cues  required  to 
guide  and  support  equipment-operation  performance.  These  cues  would 
be  depended  on  in  their  entirety  by  the  new  user,  to  a  lesser  degree  as  he 
has  practice,  until  ultimately  he  can  ignore  most  of  the  cues.  After  a 
period  of  no  practice  on  the  equipment  he  might  have  to  revert  to  use  of 
a  greater  number  of  cues. 

The  assumption  underlying  the  proposed  approach  is  that  Infor¬ 
mation  Systems  must  support  a  range  of  users  from  minimum  to  maximum 
skill,  rather  than,  as  has  been  traditional  in  equipment  design,  only  the 
user  of  maximum  skill.  Evidence  indicates  that  equipment  can  be  human- 
engineered  to  support  several  levels  of  user  proficiency.  The  major 
difference  in  behavior  during  "training"  versus  "job  performance"  thereby 
would  rest  in  the  number  of  available  cues  used  and  the  performance  time. 
The  cues,  supports  and  training  aids  which  are  proposed  here  for  availa¬ 
bility  at  the  man-machine  interface  are  referred  to  as  "self-instruc¬ 
tional  features.  "  They  range  from  such  techniques  as  color-coding  for 
guiding  fixed  procedures  to  computer-directed  programed  instruction 
built -into  the  system. 
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D.  The  Methodology 


The  first  step  in  developing  self-instructional  features  is  to  perform 
a  task  analysis.  In  its  simplest  form  the  task  analysis  may  appear  to  be 
nothing  more  than  a  very  complete  set  of  job  instructions.  For  simple 
tasks,  this  complete  set  of  instructions  may  be  all  that  is  needed  for  train¬ 
ing.  In  more  complicated  situations  the  task  analysis  will  be  used  to  spec¬ 
ify  what  capabilities  the  user  must  have  and  to  suggest  what  modifications 
should  be  made  to  the  operational  equipment  to  reduce  the  requirements 
on  the  user,  or  to  provide  self-instructional  features. 

After  the  task  requirements  have  been  determined  methods  of 
teaching  the  required  skills  or  of  modifying  the  machine  or  designing  the 
interface  to  make  its  functions  so  manifest  as  to  obviate  the  need  for  train¬ 
ing  should  be  developed.  Although  aids  may  be  developed  to  reduce  the 
total  task  training  requirements,  necessary  training  in  use  of  the  aids 
themselves  should  not  be  overlooked. 

II.  TASK  ANALYSIS 

Before  considering  the  implementation  of  teaching  functions  in  com¬ 
plex  machines  a  detailed  problem  statement  should  be  prepared.  The 
problem  statement  should  define  those  capabilities  which  will  be  required 
of  the  human  operator  for  acceptable  operational  performance  (the  termi¬ 
nal  state),  the  initial  capabilities  which  the  operator  may  be  expected  to 
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process  (the  initial  state),  and  any  intervening  levels  of  capability  which, 
when  reached,  signal  a  change  in  instructional  techniques,  aids,  or  require¬ 
ments  (the  intermediate  state(s)).  Identification  of  the  necessary  operator 
capabilities  requires  a  task  analysis.  Thus,  the  analysis  of  the  task 
characteristics  represents  an  essential  first  step  which  provides  basic 
data  for  the  problem  statement,  outlines  constraints  on  possible  solutions, 
and  may  even  suggest  the  solution.  The  task  analysis  specifies  what  capa¬ 
bilities  are  required  for  each  aspect  of  the  task,  who  must  have  these 
capabilities,  and  is  used  in  specifying  constraints  on  how  the  required 
skills  may  be  taught  using  the  operational  equipment. 

A  variety  of  task  analysis  techniques  is  available  to  the  analyst  (2,  3, 

4,  5,  6,  7,  8).  Selection  of  a  particular  method  for  analyzing,  verbally 
describing,  or  graphically  modeling  the  task  should  be  based  on  the  end 
requirements  for  the  task  analysis.  For  example,  the  analysis  of  the 
SAGE  IND  Director's  task,  described  in  a  previous  report  (1)  was  directed 
towards  the  identification  of  the  contingencies  affecting  each  action  and 
specification  of  sequences  of  actions.  Graphic  logic  flow  diagrams  and 
symbology  were  developed  and  used  to  model  these  task  characteristics. 
Manipulative  skills  and  sub-tasks  such  as  use  of  light-gun  or  contacting 
pilot  by  radio  were  not  analyzed  separately  but  were  symbolically 
represented  as  single  actions.  The  SAGE  task  analysis  was  used 
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in  developing  an  instruction  program  for  teaching  the  decision  making  as¬ 
pects  of  the  task.  A  more  detailed  task  analysis  would  have  been  required 
for  the  development  of  a  program  to  also  teach  the  manipulative  skills. 

The  task  analysis  of  the  experimental  "numbers  game",  described  in  a 
previous  report  (1)  outlined  an  "optimum  strategy"  for  obtaining  the  solu¬ 
tion.  Selection  of  each  successive  test  number  was  based  on  specifiable 
contingencies  relating  to  previous  test  numbers.  From  the  task  analysis 
those  concepts  and  bookkeeping  techniques  which  the  operator  should  be 
taught  were  easily  identified. 

In  general,  task  analyses  used  in  developing  instructional  programs 
must  identify  all  information  inputs  to  the  operator,  all  actions  that  the 
operator  may  take,  all  decisions  the  operator  must  make,  and  the  con¬ 
tingencies  which  determine  each  action  or  decision.  Each  element  in  the 
task  can  then  be  analyzed  for  the  following  operator  requirements: 

a.  understanding  of  symbols  and  prior  knowledge  of  other  data, 

b.  perceptual  skills 

c.  organization  and  interpretation  of  these  skills  and  knowledge 
into  task  performance, 

d.  manipulative  skills. 

Operator  requirements  should  be  specified  in  terms  of  acceptable 
performance  criteria,  such  as  error  rate,  time  to  completion,  allowable 
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tolerance,  etc. 

EL  PROGRAMED  INSTRUCTION  WITH  OPERATIONAL  EQUIPMENT 

After  the  task  has  been  analyzed  and  the  operator  requirements 
identified,  methods  of  providing  operators  with  the  required  capabilities 
may  be  evaluated.  Alternative  methods  include  such  possibilities  as: 
selection  of  operators  possessing  the  required  skills,  simplification  of 
the  task  so  that  instruction  is  not  required,  and  teaching  the  required  skills 
to  the  operator  trainees.  In  practice,  a  combination  of  all  three  techniques 
is  generally  used.  The  designer  of  Air  Force  Information  Systems  equip¬ 
ment  may  have  only  limited  knowledge  of  the  skills  that  the  potential 
users  will  possess  and  be  severely  restricted  as  to  the  degree  to  which 
the  task  can  be  economically  simplified.  Consequently,  consideration  of 
instructional  requirements  and  techniques  will  be  appropriate  for  most 
systems. 

Components  used  in  a  conventional  teaching  machine  are  outlined  in 
Figure  1.  Overlap  in  characteristics  of  teaching  machine  components 
with  those  of  the  operational  equipment  can  be  exploited  in  implementing 
self-teaching  functions  as  outlined  in  Figure  2.  The  displays  and  controls 
of  operational  equipment  are  often  used  at  present  for  training  and  evalua¬ 
tion  exercises,  although  instructors  or  referees  may  be  needed  for  response 
evaluation  and  reinforcement  purposes.  In  other  situations  access  to  the 
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LEARNER 


Figure  1.  Components  of  Conventional  Programed 
Instruction  Equipment  (adapted  from  Stolurow*) 
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Figure  2.  Information  System  with  Built-in 
Programed  Instruction  Subsystem 
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display  generating  equipment  and  computer  program  may  permit  use  of 
"canned"  training  programs  in  which  response  evaluation  is  performed 
automatically  and  immediate  reinforcement  provided. 

Most  Air  Force  Information  Systems  now  in  development  have  inherent 
data-processing  capability  which  can  be  harnessed  to  supply  computer- 
directed  instruction.  For  the  473L  Air  Force  Control  System,  an  oper¬ 
ational  specification  (9)  has  been  designed  to  provide  programed  instruc¬ 
tion  in  the  form  of  active  on-console  training  in  writing  and  processing 
query  language  statements.  This  computer-directed  program  will 
tailor  training  to  the  proficience  level  of  each  trainee.  Trainees  will 
compose  and  enter  queries  directly  into  the  operational  console  and  will 
be  provided  feedback  by  the  computer  program  on  the  operational  display. 

Thus,  given  considerable  skill,  ingenuity,  and  time  on  the  part  of  an 
instructional  programer  and  a  computer  programer,  it  is  possible  now 
to  provide  computer-directed  programed  instruction  in  Information 
Systems.  However,  to  make  this  efficient  training  method  economically 
as  well  as  technically  feasible  for  wide  application,  many  hardware  and 
software  problems  must  be  resolved  (11). 

Those  aspects  of  programed  instruction  which  have  implications  for 
equipment  design  parameters  include:  a)  the  necessity  for  immediate  feed¬ 
back  of  results,  b)  the  active  responses  by  the  operator  should  proceed  in 
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small  steps  so  that  he  makes  few  errors  in  the  course  of  mastering  the 
task,  c)  reinforcement  techniques  should  be  used  to  strengthen  the  learning 
(in  some  situations  knowledge  of  results  may  provide  adequate  reinforce¬ 
ment),  and  d)  instruction  should  proceed  at  a  rate  governed  by  the  rate  of  acquisi¬ 
tion  by  the  operator.  In  some  cases  it  may  not  be  possible  to  include  all 
of  the  above  features  except  at  excessive  cost.  In  such  cases  it  would  be 
desirable  to  have  sufficient  data  for  evaluating  cost-effectiveness  of  vari¬ 
ous  trade-offs  and  modifications.  Such  data  are  not  readily  available. 

Further  experiments  are  being  conducted  to  develop  general  procedures 
and  data  for  analyzing  economic  trade-off  with  instructional  efficiency. 

The  first  three  of  the  above  listed  considerations  all  relate  in  a 
general  way  to  characteristics  of  the  reinforcement  given  the  learner 
(i.  e.  that  which  modifies  his  behavior).  The  fourth,  rate  of  instruction, 
is  important  in  that  it  is  ineffective  if  too  fast,  and  inefficient  if  too  slow. 

Some  of  the  conditions  which  affect  the  reinforcing  properties  of  a  stimu¬ 
lus  are  described  in  the  following  paragraphs. 

a.  The  learner  must  understand  the  information  conveyed  by  the 
stimulus.  This  understanding  may  have  to  be  taught,  particularly  when 
feedback  of  system  status  is  used  as  the  reinforcer  (knowledge  of  results). 

For  example,  in  the  initial  model  of  the  SAGE  IND  Director's  task  in 
this  project  the  change  in  the  light  display  was  used  as  a  signal  to  the 
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learner  that  he  had  completed  the  button  sequence  correctly.  However, 
the  learner  first  had  to  be  taught  to  understand  the  meaning  of  a  light  dis¬ 
play  change  before  it  fulfilled  this  function.  A  number  of  common  signals 
have  sufficient  association  value  to  permit  correct  reinforcing  interpreta¬ 
tion  without  instruction.  Aversive  stimuli  such  as  red  lights,  raucous 
buzzers,  "ERROR"  signs,  etc.  generally  indicate  a  mistake  in  perfor¬ 
mance.  Common  stimuli  which  convey  information  that  behavior  is  cor¬ 
rect  include  green  lights,  "RIGHT"  signs,  and  production  of  the  desired 
end  result  (for  example,  indicator  on  the  timing  dial  moves  when  the 
tuning  knob  is  turned).  When  the  evaluation  of  behavior  is  more  sophis¬ 
ticated  than  the  binary  decision  "right"  or  "wrong",  indicators  such  as 
scales,  dials,  intensity  of  color  signals,  or  similar  graded  displays 
may  be  used.  This  would  be  particularly  applicable  in  training  behavior 
such  as  is  required  in  the  "numbers  game",  where  a  learner's  response 
(i.  e.  ,  the  test  number  selected)  may  be  evaluated  as  better  or  worse 
than  other  possible  responses,  but  not  absolutely  correct  or  incorrect. 

b-  The  reinforcement  must  occur  after  the  response  with  a  mini¬ 
mum  time  delay.  It  has  been  shown  that  the  effectiveness  of  a  reinforcer 
in  strengthening  desired  behavior  decreases  as  the  time  interval  between 
the  response  and  the  reinforcement  increases,  even  if  no  other  (overt) 
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behavior  intervenes.  Hence,  the  desired  achievement  is  "immediate" 
reinforcement  or  feedback  of  results.  For  continuous  tasks,  such  as 
continuous  control  tracking,  it  is  inherently  difficult  to  provide  effective 
reinforcement  directly  related  to  the  response  (the  response  is  continu¬ 
ously  changing  and  the  reinforcement  may  not  be  perceived  as  associated 
with  the  appropriate  response  characteristics). 

c.  The  reinforcement  may  be  positive,  negative,  or  combination  of 
the  two  types.  However,  excessive  or  improper  use  of  one  type  or  the 
other  may  destroy  the  reinforcing  properties  of  the  stimulus,  and  may 
even  hinder  the  learning  process.  In  general,  positive  reinforcement 
(agreeable  stimuli  or  rewards)  is  more  effective  than  is  negative  rein¬ 
forcement  (aversive  stimuli  or  punishment)  in  molding  behavior.  As 
the  learner  or  user  masters  the  task  and  becomes  "sure"  of  himself 
some  forms  of  stimuli  which  continuously  tell  him  that  "You're  right" 
may  become  a  nuisance.  In  using  positive  reinforcement  when  learning 
through  use  of  operational  equipment  it  would  be  desirable  to  substitute 
knowledge  of  results  in  terms  of  system  status  in  place  of  artificial  rein¬ 
forcers  as  the  operator  learns  the  meaning  of  system  status  variables 
(see  discussion  on  schedules  of  reinforcement).  An  excessive  use  of 
negative  reinforcement  may  also  hinder  learning  by  creating  a  frustrated 
or  negative  attitude.  If  the  task  cannot  be  taught  in  sufficiently  small 
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steps  so  that  errors  are  infrequent,  the  value  of  positive  reinforcement 
will  increase  and  particularly  aversive  negative  reinforcers  should  not 
be  used. 

d.  There  are  no  firm  rules  regarding  the  quantity  or  degree  of 
physical  characteristics  which  should  be  used  for  various  types  of  rein¬ 
forcement.  If  the  reinforcement  is  money,  the  quantity  is  important. 
Optimum  quantity  will  vary  with  the  type  of  reinforcer,  frequency  of 
anticipated  correct  and  incorrect  responses,  and  reinforcement  history 
of  the  particular  learner.  When  using  lights  as  reinforcers  the  intensity 
and  duration,  for  example,  should  be  selected  to  insure  both  attention 
getting  and  reinforcing  properties. 

e.  The  schedule  of  reinforcement  should  be  varied  according  to  the 
state  of  learning  which  has  been  achieved.  No  general  answer  can  be 
given  to  the  question,  "When  shall  reinforcement  be  administered?". 

The  previously  described  experiments  with  the  console  model  of  the 
SAGE  IND  Director's  task  demonstrated  that  initially  it  was  important  to 
give  reinforcement  often,  to  guide  the  operator  in  developing  hypotheses 
about  what  the  task  really  was.  Unreinforced  responses  were  difficult 
for  the  subject  to  relate  to  his  hypotheses,  and  thereby  delayed  or  re¬ 
duced  learning.  However,  after  a  period  of  training,  the  subject  will 
have  learned  short  sequences  of  actions  sufficiently  to  consider  the 
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sequences  as  individual  units  or  "chunks".  The  most  efficient  reinforce¬ 
ment  schedule  is  then  to  reinforce  only  after  the  response  "chunks"  are 
emitted.  Considerable  logic  and  memory  capability  is  required  for  a 
machine  to  decide  when  chunking  has  occurred,  and  what  changes  in  rein¬ 
forcement  schedules  should  be  made.  When  operators  receive  their  initial, 
intermediate,  and  advanced  instruction  by  using  the  actual  operational 
equipment,  provision  for  altering  the  reinforcement  schedule  should  be 
included. 

The  above  considerations  deal  with  the  technical  decisions  that  the 
designer  must  make  in  implementing  instructional  features  in  operational 
equipment.  The  present  study  has  not  been  sufficiently  detailed  to  permit 
the  drawing  of  conclusions  regarding  the  economic  decisions  that  the  de¬ 
signer  must  make.  Inclusion  of  self-instructional  features  generally  in¬ 
volves  increased  capital  equipment  and  design  costs.  In  addition,  the  com¬ 
plexity  of  the  system  may  or  may  not  be  increased  with  increased  main¬ 
tenance  costs.  Consequently,  the  designer  must  relate  each  technical 
decision  to  the  economic  realities  governing  system  design.  A  detailed 
task  and  training  analysis  provides  a  powerful  tool  for  economic  analysis 
of  system  design  factors.  Such  an  analysis  will  indicate  the  actual  costs 
of  having  a  man  perform  various  functions  within  the  system,  as  well  as 
indicating  where  operator  load  may  be  increased  if  significant  equipment 
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or  personnel  savings  could  be  realized. 

IV.  SEQUENCING  AIDS 

If  the  task  analysis  indicates  that  the  order  in  which  actions  are  taken 
is  important  to  system  performance,  then  consideration  of  sequencing  aids 
is  appropriate.  Highly  developed  sequencing  aids  may  simplify  the  task 
sufficiently  to  eliminate  the  need  for  learning  the  sequences.  However, 
in  other  situations  the  aids  will  assist  the  user  in  proper  sequencing  during 
his  training  period  and  then  be  ignored  when  training  is  complete.  Princi¬ 
pal  categories  of  sequencing  aids  for  Information  System  equipment  include: 

a.  color  coding  of  controls,  diagrams,  or  panels 

b.  flow  charts  or  diagrams 

c.  spatial  layout  of  console  controls 

d.  word  cueing  (visual  or  acoustic) 

e.  alphanumeric  or  symbolic  coding  and  cueing. 

Since  the  effectiveness  of  the  sequencing  aid  is  dependent  on  the  user's 
understanding,  requirements  for  teaching  the  symbology  and/or  terminology 
of  the  aid  should  also  be  evaluated.  For  example,  it  was  necessary  to  teach 
the  symbology  of  the  logic  flow  diagrams  used  in  the  console  model  SAGE 
END  Director's  task  which  was  previously  described.  Instruction  was 
necessary  even  though  the  symbology  was  simple  and  logical  (i.e.  ,  buttons 
along  a  color  coded  line  proceeding  from  left  to  right  indicated  an  ordered 
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"and"  sequence,  while  buttons  spatially  oriented  one  above  another  and  all 
enclosed  in  a  color  coded  box  indicated  an  unordered  "and"  sequence). 

The  categories  of  actions  included  in  sequences,  the  types  of  sequences, 
and  the  contingencies  associated  with  "branch"  points  should  be  considered 
in  selecting  sequencing  aids.  For  example,  obtaining  information  from  a 
data  processing  system  may  involve  activating  a  selector  switch,  use  of 
a  light  gun  for  target  identification,  and  then  activation  of  a  switch  to 
obtain  a  readout  of  the  requested  information.  If  the  two  switches  and 
the  light  gun  are  used  only  in  this  sequence,  the  two  switches  might  be 
placed  on  either  side  of  the  light  gun  rack,  all  coded  the  same  color,  and 
labeled  1,  2,  3  in  order  of  use.  If  other  forms  of  readout  are  used,  de¬ 
pending  on  specified  contingencies,  the  appropriate  readout  switch  might 
better  be  located  adjacent  to  the  alternative  readout  switches,  and  all  such 
switches  coded  according  to  the  contingent  display.  Sequencing  from  the 
light  gun  to  alternative  readout  switches  might  then  be  aided  by  arrows 
printed  on  the  console  panel  leading  from  the  light  gun  rack  to  the  group 
of  switches.  Alternatively  a  lucite  strip  around  the  switches  might  be 
illuminated  whenever  the  light  gun  was  activated. 

Characteristics  of  a  number  of  sequencing  aids  have  been  outlined  in 
the  following  paragraphs.  Consideration  of  these  characteristics  may 
provide  guidelines  for  specifying  aids  for  particular  systems. 
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a  Color  Coding:  Although  only  a  very  limited  number  of  solid 
colors  may  be  rapidly  discriminated,  through  use  of  stripe  or  checker¬ 
board  patterns  several  dozen  distinctive  codes  can  be  used.  Logic  flow 
diagrams  for  the  console  model  of  the  SAGE  IND  Director’s  task  employed 
thirteen  color  codes  (ten  solid  color  and  three  stripe  patterns).  The  stripe 
patterns  were  observed  to  be  equally  or  more  discriminable  than  correspond¬ 
ing  solid  color  codes.  When  viewing  conditions  are  not  optimal  (i.  e.  low 
levels  of  illumination,  glare  sources,  colored  ambient  lighting,  etc. ) 
colors  must  be  selected  with  greater  care  to  avoid  use  of  poorly  differ¬ 
entiated  codes.  Color  codes  can  be  applied  to  individual  controls,  console 
panel  areas,  or  to  panel  flow  charts.  Use  of  colors  with  high  association 
values  may  facilitate  training.  For  example,  emergency  controls  might 
be  red,  or  located  on  a  red  panel.  However,  the  association  value  of 
colors  is  highly  dependent  on  the  individual’s  past  experience  -  red  and 
green  might  indicate  acetylene  and  oxygen  controls  to  a  chemist  -  and  care 
should  be  taken  to  avoid  situations  in  which  undesirable  associations 
could  increase  error  during  operation. 

Color  codes  have  been  widely  used  to  indicate  categories  of  con¬ 
sole  controls  (for  example,  all  display  controls  of  one  color,  and  all  com¬ 
munications  controls  of  another  color).  Experiments  with  the  abstract  con¬ 
sole  model  of  the  SAGE  IND  Director's  task  have  demonstrated  that  color 
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codes  also  work  well  for  flow  diagrams.  Wasserman  (12)  has  also 
demonstrated  that  four  color  coding  of  circuit  diagrams  is  superior  to 
four  types  of  lines  used  as  codes  in  terms  of  speed  of  following  and  num¬ 
ber  of  errors. 

b.  Flow  Charts  or  Diagrams:  Flow  charts  allow  rapid  following  of 
sequences  of  actions,  provided  the  operator  understands  the  logic  symbol¬ 
ogy  used.  If  sequences  of  use  of  controls  and  displays  can  be  specified  in 
advance,  diagrams  or  overlays  can  be  placed  on  the  panel  with  arrows 
and/or  other  high  association  value  or  learned  symbols.  The  flow  charts 
may  be  color  coded  to  avoid  errors  where  paths  branch  or  cross,  although 
identification  of  contingencies  by  color  codes  may  require  additional  instruc¬ 
tion.  Active  flow  charts,  such  as  may  be  constructed  using  illuminated 
lucite  bars  or  overlays,  may  be  particularly  useful  in  instructing  sequen¬ 
tial  activity,  but  means  of  identifying  the  appropriate  sequence  for  activa¬ 
tion  may  be  more  difficult  to  implement.  However,  if  logic  circuits  are 
provided  to  guide  the  operator  from  control  to  control  by  means  of  lights 
or  illuminated  strips,  the  same  circuits  may  also  be  useful  in  implementing 
reinforcing  stimuli  at  small  additional  cost. 

In  using  flow  charts  as  sequencing  aids  it  is  important  to  identify  the 
starting  point,  direction  in  which  to  proceed,  points  where  contingencies 
should  be  checked,  ordered  versus  unordered  sequences,  and  to  utilize 
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symbology  which  has  a  high  association  value.  Restricted  flow  charts  can 
be  used  wherever  sequence  can  be  specified,  even  though  the  particular 
sequence  represents  only  a  portion  of  the  task.  For  example,  if  display 
focus  is  always  adjusted  after  display  gain  is  altered,  an  arrow  from  the 
gain  to  the  focus  control  is  appropriate. 

Flow  charts  used  in  teaching  sequences  can  be  projected  onto  consoles 
when  overlays  or  permanent  flow  charts  are  inappropriate.  The  slide 
projector  may  be  synchronized  to  a  taped  lecture  which  explains  task 
components,  contingencies,  and  procedures.  A  servo  controlled  light 
pointed  could  also  be  used  to  trace  out  the  sequences  of  actions.  Use  of 
these  techniques  have  the  advantage  of  not  interferring  with  the  design  of 
the  operational  equipment,  and  yet  allowing  the  learner  to  obtain  his  train¬ 
ing  in  the  actual  operational  environment. 

In  discussing  instruction  through  display  of  equipment  operated  se¬ 
quences  the  following  distinctions  should  be  made:  a)  passive  display  of 
operations  may  consist  of  slides,  diagrams,  etc.  which  the  user  reviews 
at  his  own  pace,  b)  active  display  of  operations,  such  as  through  taped 
lecture  and  automatic  pointer,  involves  a  forced  pace. 

The  disadvantage  of  the  display  teaching  technique  is  that  the  trainee  is 
not  required  to  make  an  overt  response.  It  would  be  possible  in  some  situa¬ 
tions  to  devise  a  system  whereby  a  manipulator  would  grab  the  user's 
hand  and  force  the  user  to  follow  through  the  sequences.  A  simpler 
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system  might  involve  having  the  trainee  manipulate  a  light  pointer  to  "track" 
an  automated  pointer,  and  then  measure  the  "errors".  These  approaches 
are  in  contrast  to  the  Skinnerian  "free  operant"  approach  which  would  be 
to  instruct  the  trainee  minimally,  allow  the  trainee  to  perform  the  opera¬ 
tions  in  large  segments,  measure  or  evaluate  the  performance,  then  accord¬ 
ingly  reinforce  the  behavior;  any  partially  correct  response  would  be  re¬ 
warded  at  first,  but  gradually  the  class  of  rewarded  responses  would  be 
restricted. 

c.  Spatial  Layout:  Where  sequences  can  be  specified  in  advance 
spatial  layout  of  controls  will  be  effective  (i.e.  proceed  from  left  to  right, 
top  to  bottom).  If  controls  are  used  several  times  in  a  sequence,  spatial 
layout  will  require  duplication  of  controls,  or  a  combination  of  flow  dia¬ 
grams  and  partial  spatial  layout.  The  natural  tendency  of  the  human  oper¬ 
ator  to  proceed  from  left  to  right  and  top  to  bottom  can  often  be  advan¬ 
tageously  used  for  layout  of  subsets  of  sequenced  controls. 

d.  Word  Cueing:  Word  cueing  (visual  or  auditory)  can  be  used  in  a 
variety  of  forms  for  instruction  in  the  sequencing  of  actions.  In  its  simplest 
form  a  set  of  written  instructions  can  be  given  to  the  operator,  automatically 
displayed,  or  printed  on  the  console  panel.  In  complex  systems  in  which 
computer  programming  for  a  "training  mode"  can  be  implemented  it  may 

be  possible  to  display  worded  instructions  after  each  step  in  the  solution 
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of  a  "canned"  problem.  Even  if  the  optimum  sequence  cannot  be  specified 
beforehand,  general  guiding  instructions  may  be  displayed  according  to 
an  acceptable  strategy  (both  the  form  and  content  on  instructions  are  im¬ 
portant  to  effectiveness,  and  guiding  comments  may  not  lead  to  the  best 
or  most  rapid  learning  of  querying-reasoning  sequences,  as  demonstrated 
in  the  "numbers  game"  experiments). 

In  using  word  cueing  the  selection  of  words  may  be  restricted  by  the 
amount  of  prior  training  of  the  operator.  Labeling  of  controls  will  not  in 
itself  aid  sequencing. 

e.  Alphanumeric  or  Symbolic  Coding  and  Cueing:  Alphanumeric 
coding  is  particularly  useful  in  designating  sequences.  People  are 
accustomed  to  proceding  through  the  alphabet  from  a  to  z,  and  through  an 
array  of  digits  according  to  increasing  numerical  value.  More  complex 
codes  combining  alphabetic  and  numeric  characters  can  be  devised  and 
taught  to  the  users.  Such  codes  can  be  used  alone  or  to  complement 
color  or  symbolic  codes  . 

Symbolic  codes  can  be  devised  which  greatly  increase  the  utility 
of  logic  flow  diagrams  as  sequencing  aids.  Alphanumeric  information 
added  to  the  symbols  used  in  flow  charts  has  been  used  to  specify  se¬ 
quences  of  actions  and  governing  contingencies  in  the  SAGE  IND  Director's 
task  (1).  However,  it  was  determined  that  users  did  require  training  in 
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interpreting  the  symbology  before  they  could  properly  sequence  actions 
by  following  the  flow  chart. 

V.  QUERYING  -REASONING  AIDS 

The  successive  refinement  of  hypotheses  about  (the  cause  of)  data, 
given  successive  presentations  of  data  known  to  be  from  one  of  a  given 
set  of  causes,  may  be  regarded  as  learning.  Consequently,  devices  which 
assist  in  selecting  appropriate  data  requests,  integrating  present  data 
with  past  data,  and  computing  or  inferring  the  probability  of  each  of  a 
set  of  hypotheses  may  be  used  as  instructional  aids.  The  human  opera¬ 
tor  appears  to  be  good  at  making  judgements  about  effects  (data)  given 
causes  (hypotheses)  and  poor  at  judging  causes  given  effects.  Presum¬ 
ably  this  is  because  of  an  ability  in  "thought  experimentation"  or  "gedanken 
experiments",  (fabrication  of  a  mental  model  of  a  process  for  a  given  cause 
or  hypothesis)  and  not  because  of  ability  to  do  mental  bookkeeping.  Aids 
or  instruction  in  bookkeeping  or  in  determining  causes,  given  effects, 

(as,  for  example,  by  use  of  Bayes'  theorem)  should  have  a  high  potential 
for  improving  performance. 

In  the  "numbers  game"  experiments  (1)  several  forms  of  instruction 
were  evaluated.  In  the  initial  stages  of  learning  an  automated  bookeeping 
function  provided  the  greatest  amount  of  performance  improvement. 

When  the  automated  bookkeeping  was  discontinued  a  sharp  performance 
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decrement  was  observed,  but  most  subjects  soon  adopted  manual  book¬ 
keeping  techniques  similar  to  those  which  had  previously  been  automated. 
The  use  of  the  automated  bookkeeping  thus  appeared  to  act  both  as  a  means 
of  task  simplification  and  as  an  instructional  technique.  It  is  anticipated 
that  implementation  of  simple  logic  and  bookkeeping  tasks  in  complex 
Information  System  equipment  may  greatly  reduce  the  amount  of  instruc¬ 
tion  required.  Operators  who  have  been  trained  initially  on  equipment 
which  automatically  performs  much  of  the  routine  logic  and  bookkeeping 
operations  should  require  only  minimal  reinstruction  when  called  upon  to 
perform  the  same  task  with  less  automated  equipment  in  the  field. 

Guidance  in  querying-reasoning  behavior  may  be  provided  through  a 
display  of  "Socratic"  comments  after  each  action.  However,  there  is 
some  evidence  (our  experience  with  the  numbers  game)  that  such  comments 
may  not  be  effective  for  all  subjects  during  the  early  phases  of  instruction 
(Socrates  used  this  technique  with  fairly  advanced  students).  Such  com¬ 
ments  may  be  useful  primarily  in  guiding  advanced  trainees  in  understand¬ 
ing  the  fine  nuances  of  optimal  strategies. 

VI.  CONCLUSION 

Varying  degrees  of  self-instructional  capabilities  can  be  included  in 
Air  Force  Information  System  equipment.  However,  there  are  many  un¬ 
known  factors  associated  with  the  economics  of  such  features.  Before 
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proceeding  with  design  modifications  a  detailed  task  analysis  should  be 
performed  to  identify  the  required  operator  skills.  After  the  operator 
requirements  have  been  identified  for  each  decision  or  action  then: 
a)  the  sequences  should  be  examined  for  possible  inclusion  of  simplifica¬ 
tions  or  sequencing  aids,  b)  contingency  analysis  and  bookkeeping  require¬ 
ments  should  be  examined  for  possible  simplification  through  automation, 
and  then  c)  instructional  requirements  should  be  derived  by  comparing 
the  remaining  required  skills  to  those  of  operator  trainees. 

Several  techniques  and  considerations  relating  to  the  aiding  of  proper 
sequencing  of  actions,  logic  analysis,  and  querying-reasoning  behavior 
have  been  described.  Use  of  seme  of  these  techniques  will  reduce  instruc¬ 
tional  requirements,  others  are  designed  to  teach  the  required  skills  when 
the  operator  trainee  receives  his  initial  instruction  on  the  operational 
equipment.  Through  the  use  of  portions  of  the  operational  equipment,  or 
the  entire  system,  conventional  programed  instruction  features  can  be 
provided  with  a  minimum  of  additional  equipment.  However,  consideration 
of  such  inclusion  should  occur  early  in  the  design  process  so  that  sufficient 
flexibility  to  meet  changing  instructional  requirements  may  be  assured. 
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